Harvard University announced on Thursday the release of a high-quality dataset containing nearly one million public domain books, which anyone can use to train large language models and other AI tools. This dataset was created by Harvard's newly established Institutional Data Initiative and is funded by Microsoft and OpenAI. The included books are all works scanned by the Google Books Project that are no longer under copyright protection.

The New York Times and the Daily News encountered an unexpected twist in their copyright lawsuit: an OpenAI engineer inadvertently deleted virtual machine search data that could have been key evidence, adding a dramatic turn to this high-profile legal dispute. According to a letter submitted to the U.S. District Court for the Southern District of New York on Wednesday night, lawyers and technical experts for the two media companies had previously invested over 150 hours searching OpenAI's AI training dataset. However, on November 14, an OpenAI engineer accidentally deleted data stored on the virtual machine.

In the context of rapid developments in artificial intelligence, Baidu Tieba's 'Stupid Bar' has unexpectedly become an important data source for AI training, challenging traditional perceptions of high-quality data. This community, known for its absurd and humorous content, has outperformed well-known platforms in tests such as Q&A and brainstorming, making it one of the most popular AI training databases. Research shows that the conversational and multi-turn Q&A data from 'Stupid Bar' helps AI come closer to human communication styles. While AI has limitations in understanding human humor, the jokes from 'Stupid Bar' feature tight logic, rich rhetoric, and strong...

LAION launched Re-LAION-5B, the world's first AI training dataset that fully removes links to CSAM, aimed at addressing the issue of Child Sexual Abuse Material (CSAM). This dataset has been significantly improved over LAION-5B and is mainly divided into two versions: Re-LAION-5B Research and Research-Safe. A total of 2,236 CSAM links have been removed, including 1,008 from child protection organizations' lists. The dataset contains 5.5 billion pairs of text and images, designed to help

Recently, the 'Diting' seismic wave large model, jointly developed by the National Supercomputing Center in Chengdu, the Institute of Geophysics of the China Earthquake Administration, and Tsinghua University, was officially released in Chengdu, Sichuan. This model is the first seismic wave large model in the country to reach 100 million parameters, marking a significant breakthrough in the integration of seismology research and artificial intelligence technology in China.

Spawning is committed to giving artists more control over the online use of their work. The company has launched its latest project called Source.Plus, aimed at screening 'non-infringing' media content for AI model training, enabling artists and creators to more precisely manage the usage rights of their works.

OpenAI has launched the Data Partnerships initiative to collaborate with third parties in creating public and private datasets for AI model training. OpenAI seeks data across multiple domains, languages, and formats to enhance the understanding of AI models. The initiative aims to collect various forms of data, including images, audio, and video, to reflect human society and achieve the goal of safer, more beneficial AI for humanity. OpenAI's initiative has sparked discussions about the rights of data owners and transparency.

The AI industry is facing a crisis in training data that may slow down the growth of AI models. High-quality data is crucial, but it may run out before 2026. Solutions include improving algorithms and using synthetic data.

The anti-piracy organization successfully demanded the takedown of the AI training dataset 'Books3' from the online piracy book repository The Eye. The 'Books3' dataset consists of 37GB of text, which has been used to train artificial intelligence models by multiple companies. The anti-piracy organization stated that AI poses new challenges to copyright and plans to continue combating other websites hosting this dataset.

Google and OpenAI use bots to collect data from the internet, but recent copyright laws have posed obstacles to their data scraping efforts. AI models require human-generated content to improve, but whether companies should pay for or freely obtain this content is a concern. OpenAI has begun using datasets created by ChatGPT to train GPT-4, but relying solely on this data could lead to model failures.

['Multiple media organizations urge the establishment of rules to protect the copyright of data used for training generative AI models.', 'Calling on global lawmakers to create regulations requiring transparency in training datasets and obtaining consent from rights holders.', 'Demanding that AI companies eliminate bias and misinformation and identify the sources of generated content.']